The popularity of the Internet and other private and public networks has resulted in increasing reliance on electronic transmission of data for both business and personal reasons. In many instances the data transmitted is of a personal or confidential nature. A user may provide credit card or other account information over the network to purchase goods or services. Sensitive and confidential business communications, emails, and documents are also transmitted over networks. It is necessary to employ security measures to protect the security of such data routed over the networks.
A popular protocol for electronically transmitting data is the Internet Protocol (IP). However, IP provides little security as the data within IP packets is not encrypted and may be accessed, viewed, and even altered by an eavesdropper. Thus, IP does not protect the confidentiality or authenticity of the data. To address these shortcomings, a set of extensions to IP, referred to as the Internet Protocol Security (IPSec) suite, were developed by the Internet Engineering Task Force (IETF). The IPSec suite includes protocols such as authentication header (AH), encapsulating security protocol (ESP), and key management and exchange protocol (IKE).
The ESP protocol, documented mainly in IETF Request for Comments (RFC) 2406, is an authenticating and encrypting protocol that uses cryptographic mechanisms to provide integrity, source authentication, and confidentiality of data. The AH protocol, documented mainly by IETF RFC 2402, is an authentication protocol that uses a hash signature in the packet header to validate the integrity of the packet data and authenticity of the sender.
The IKE protocol, documented mainly in IETF RFC 2409, provides a method for an initiating computer, referred to as an initiator, and a responding computer, referred to as a responder, to negotiate security settings used with the AH and ESP protocols. The negotiated security settings form a data structure called a security association (SA). The SA defines parameters such as an authentication algorithm, encryption algorithm, keys, and the lifetime of keys, used by ESP or AH to protect the contents of the IP packet. Because ESP and AH require an established SA, an IKE negotiation is executed before the ESP or AH protocols are used by the initiator and responder.
The IKE negotiation includes two phases. In a first phase, known as “main mode,” the initiator and the responder establish a secure channel for conducting IKE negotiation, referred to as an IKE SA. In the second phase, known as “quick mode,” the initiator and the responder negotiate general purpose SAs over the secure channel established in the first phase. The main mode and quick mode phases of the IKE negotiation each include a series of IKE packets exchanged between the initiator and responder. According to the IKE protocol, each IKE packet includes a User Datagram Protocol (UDP) header with a source and a destination port address. Typically, the UDP header includes source and destination ports of 500 as that is the Internet Assigned Numbers Authority (IANA) port for IKE packets.
The IKE main mode negotiation begins when the initiator sends an initial IKE request to the responder that includes an SA with proposed parameters. The responder responds with an SA that includes the parameters to which the responder agrees. The initiator and responder then exchange a second pair of messages that include key exchange data for a Diffie-Hellman key exchange. Finally, the initiator and responder exchange a third pair of messages, which are used to authenticate the initiator and responder. The IKE protocol requires the responder to maintain state, i.e. store information, such as the proposed and agreed upon SA parameters throughout the entire main mode negotiation process. Thus, each ongoing IKE negotiation consumes responder resources.
Recent proposals suggest methods that may be used to implement the IPSec protocol suite in network environments where a network address translation device (NAT) is employed. An example of such a network environment includes a NAT connected to a plurality of network devices over an internal network, such as a Local Area Network (LAN), as an interface between the internal network and an external network, such as the Internet. Each network device has a private source IP address that is valid on the internal network but not valid on the external network. When a first network device sends an IP packet destined for the external network, the NAT intercepts the packet and replaces the private source IP address with an IP address valid on the external network, such as a public IP address assigned to the NAT. The NAT performs the same process for each network device, and in each case uses the same source public IP address. The use of NAT advantageously allows multiple network devices to communicate over the external network with a single source IP address.
The NAT may also change a source port in a transport layer header, e.g. TCP or UDP header, within the IP packet to ensure that each initiator sends packets with a unique combination of IP addresses and ports over the external network. Thus, when the initiator is behind a NAT and sends an IKE packet destined for a responder on the external network, the NAT typically changes the source port from 500 to some other value, which may be selected from one of over 64,000 ports. The unique combination of source port and IP addresses provides the NAT with a mechanism to route response IP packets sent from the external network to the proper network device on the internal network.
The IKE main mode negotiation process provides a malicious user with an opportunity to mount denial-of-service (DOS) attacks on a responder. For example, a malicious user can send a large number of IKE requests with spoofed, i.e. false, IP addresses. As previously described, each IKE request consumes responder resources because the IKE protocol requires the responder to maintain state throughout the IKE main mode negotiation. By sending a sufficient number of IKE requests, the attacker can monopolize the responder's resources and prevent or substantially limit the its ability to process IKE requests from legitimate network devices.
The implementation of IPSec in NAT environments creates even further opportunities for an attacker to mount a DOS attack during the IKE negotiation process. Because of NAT, the responder receives IKE requests with UDP headers that include the range of possible source ports and not just port 500. Moreover, proposals exist for using destination ports other than 500 in the IKE packet UDP header, such as described in copending U.S. patent application Ser. No. 10/284,931 entitled “Method and Apparatus for Traversing a Translation Device with a Security Protocol” thereby increasing the number of port combinations that can be used. Thus, even if the responder limits the IKE requests for which it maintains state to IKE requests sent from a device using its actual IP address, a malicious user can still flood the responder with IKE requests from varying ports thereby mounting a DOS attack.
From the foregoing, it is apparent that a new method is needed to protect a responder from DOS attacks conducted during key exchange protocol negotiations.